Tomas Kontrimas, Elena Manao, Dr. Martin Wolf
Advanced Python Programming, WS 2023/24
class Word:
def __init__(self, word):
self.word = word
def __add__(self, rhs):
return Word(self.word + ' ' + str(rhs))
def __str__(self):
return self.word
hello = Word('Hello')
world = Word('World')
print(hello + world)
Hello World
git and github.comComputer with Python and Jupyter Notebook support
Alternatively, create and use Jupyter notebooks on google: https://colab.research.google.com
Microsoft Visual Studio Code is a integrated developer environment (IDE) supporting Python and Jupyter notebooks as well.
github.com account
For most of our professional time we work with ...
All of the above needs software that is ...
Example: Trigger rate of the IceCube Neutrino Detector at South Pole:

git?This shows the installation of Jupyter and Python virtual environments on Linux (Ubuntu).
First, let's install Jupyter via apt-get:
sudo apt-get install python3-notebook python3-ipykernel
For this course, I recommend to setup a dedicated virtual Python 3 environment:
sudo apt-get install virtualenvwrapper
source /usr/share/virtualenvwrapper/virtualenvwrapper.sh
The environment is created via:
mkvirtualenv -p /usr/bin/python3 --system-site-packages basic-py3-course
pip install -U astropy ipympl matplotlib numpy scipy
The final step is to add the virtual Python environment to Jupyter Notebook:
python -m ipykernel install --user \
--name basic-py3-course --display-name "Python 3 (basic-py3-course)"
I further recommend to put the
source /usr/share/virtualenvwrapper/virtualenvwrapper.sh
line into your .profile and launch a new terminal window.
You can start a Jupyter Notebook via:
jupyter notebook
and create a new basic-py3-course notebook.
For executing Python scripts, you can activate the basic-py3-course environment via:
workon basic-py3-course
and deactivate it via:
deactivate
!pip install <package>import this
The Zen of Python, by Tim Peters Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Now is better than never. Although never is often better than *right* now. If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea -- let's do more of those!
# Comments start with '#'.
# Import modules.
import os
import matplotlib.pyplot
import numpy as np
# Import only specific functions from a module.
from scipy import interpolate, stats
# Remember: never ever do 'from foo import *'
# Declare a variable.
a = 5.
a = "Hello World!"
int, float, complexstr, unicode, list, tuple, bytearray, bufferset, frozensetdictboolNonelist and dict are copied by reference.int and float are copied by value.# Addition and subtraction
print("5. + 5. =", 5. + 5.)
print("10. - 5. =", 10. - 5.)
5. + 5. = 10.0 10. - 5. = 5.0
# Multiplication, division, and floor division
print("5. * 5. =", 5. * 5.)
print("25. / 5. =", 25. / 5.)
print("5.3 // 2 =", 5.3 // 2)
5. * 5. = 25.0 25. / 5. = 5.0 5.3 // 2 = 2.0
# Modulo and exponentiation
print("25. % 4. =", 25. % 4.)
print("5.**2 =", 5.**2)
25. % 4. = 1.0 5.**2 = 25.0
# Do calculations with variables.
a = 5.
print("a + 5. =", a + 5.)
a + 5. = 10.0
b = 4.
c = a + b
print("c = a + b =", c)
c = a + b = 9.0
# In-place modifications
a += 5.
print("a =", a)
a = 10.0
# Call mathematical functions like sin, exp, log, ...
# Instead of NumPy, you can also use the 'math' module.
print("exp(2.5) =", np.exp(2.5))
print("sin(pi/2.) =", np.sin(np.pi/2.))
exp(2.5) = 12.182493960703473 sin(pi/2.) = 1.0
General form:
[fill][align][sign][#][0][width][,][.precision][type]
See:
https://docs.python.org/3/library/string.html#format-string-syntax
# Some examples
print("{:>+10.2f}".format(np.pi))
print("{:010d}".format(3))
+3.14 0000000003
# Some more examples
s = "{0}, {1[0]}, {1[1]}, {2[a]}"
print(s.format(0, [1, 2], {"a": 3}))
print("{key}".format(key=4))
0, 1, 2, 3 4
F-string (formatted string literal) introduced in Python 3.6 supports the same string formatting form as .format().
They can embed Python expressions directly inside them using f"string example {expression:format}" syntax.
# f-string examples
a = 10.5
print(f"a = {a}")
print(f"a = {a:.0f}")
print(f"Evaluate some code: {5*5:.2f}")
print(f"Evaluate some more code: {np.sqrt(9):.0f}")
a = 10.5 a = 10 Evaluate some code: 25.00 Evaluate some more code: 3
# Example for-loop with conditions
for i in range(3):
if i == 0:
print("Skip 0.")
elif i % 2 > 0:
print("{} is an odd number.".format(i))
else:
print("{} is an even number.".format(i))
Skip 0. 1 is an odd number. 2 is an even number.
# Example while-loop
d = {"a": 0, "b": 1}
while len(d) > 0:
d.popitem()
break.continue.For loops can have an else-block, which gets
executed when the loop finished normally, i.e. without
encountering a break:
for i in [1, 2, 3]:
if i == 0:
break
else:
print('No 0 found!')
No 0 found!
for i in [0, 1, 2, 3]:
if i == 0:
break
else:
print('No 0 found!')
An easy way to create iterables from existing ones.
# List comprehension examples
l = [i**2 for i in range(10)]
l = [10. / i for i in range(10) if i > 0]
l = [10. / i if i > 0 else np.inf for n in range(10)]
l = [[a*i for a in [1, 2]] for i in range(1, 11)]
# The same works for dictionaries.
d = {"a": 2., "b": 3.}
d = {k: v**2 for k, v in d.items()}
# And of course sets
s = {i**2 for i in range(10)}
# Some generator magic
g = (i**2 for i in range(10))
r = sum(g)
# We can also do this in one line.
print(sum(i**2 for i in range(10)))
285
# Create a text file and give it some input.
with open("example.txt", "w") as stream:
stream.write("This is a line.\n")
stream.write("This is another line.")
# Read the content of the text file.
with open("example.txt", "r") as stream:
lines = stream.readlines()
print("".join(lines))
This is a line. This is another line.
import random
# Initialize the random number generator with a seed.
# Use None as seed for using the current time.
random.seed(42)
# Return the next floating point number between [0 and 1).
print(random.random())
# Return a uniformly distributed floating point number in the range [a, b].
a = 1
b = 6
print(random.uniform(a, b))
# See numpy how to generate a sequence of random numbers.
0.6394267984578837 1.1250537761133348
# A simple function that takes two arguments.
def power(a, e=2.):
r"""Exponentiation
Calculate: ``a**e``.
Parameters
----------
a : float
Some number
e : float, optional
Exponent
Returns
-------
float
Result of ``a**e``
"""
return a**e
print("5.**2 =", power(5.))
5.**2 = 25.0
# Alternative ways to call the function
results = [
power(5., 2.),
power(5., e=2.),
power(a=5., e=2.),
power(e=2., a=5.),
power(*[5., 2.]),
power(**{"a": 5., "e": 2.})
]
print("Results:", ", ".join("{}".format(r) for r in results))
Results: 25.0, 25.0, 25.0, 25.0, 25.0, 25.0
# General syntax
def func(*args, **kwargs):
print(f'args: {args}')
print(f'kwargs: {kwargs}')
func(3 ,'a', a=2, b='1')
args: (3, 'a')
kwargs: {'a': 2, 'b': '1'}
A few words about doc strings:
Lambdas are unnamed in-line functions.
# Example for lambda usage
print(sorted("String"))
print(sorted("String", key=lambda s: s.lower()))
['S', 'g', 'i', 'n', 'r', 't'] ['g', 'i', 'n', 'r', 'S', 't']
# A lambda function can be assigned to a variable.
square = lambda x: x**2
print("5**2 =", square(5.))
5**2 = 25.0
# Write a function that can raise an exception.
def division(a, b):
r"""Divide `a` by `b`.
Parameters
----------
a,b : float
Nominator and denominator
Returns
-------
float
The result of ``a/b``
Raises
------
ValueError
If the denominator `b` is zero.
"""
if np.fabs(b) > 0.:
return a / b
else:
raise ValueError("The denominator should not be zero.")
# Catch the exception.
try:
r = division(5., 0.)
except ValueError as e:
print(e)
r = np.inf
finally:
print("The result is {}.".format(r))
The denominator should not be zero. The result is inf.
A full list of built-in exceptions can be found here:
https://docs.python.org/3/library/exceptions.html
The most common exception types that one wants to raise are:
TypeErrorValueErrorNameErrorIndexErrorKeyErrorNotImplementedErrorDecorators are syntactic sugar for functions that are decorating other functions.
# Example of a simple decorating function.
def paragraph(f):
return lambda name: "<p>{}</p>".format(f(name))
@paragraph
def greet(name):
return "Hello {}, how are you?".format(name)
print(greet("John"))
<p>Hello John, how are you?</p>
Python allows to define functions within functions.
def preparing_to_feed_cat(what_to_feed):
def feed_cat(how_much_to_feed):
msg = f"I'm feeding my cat with {how_much_to_feed} {what_to_feed}."
return msg
return feed_cat
# Create a function object that knows what to feed the cat with.
feed_cat_func = preparing_to_feed_cat(what_to_feed='cans of meat')
print(feed_cat_func(how_much_to_feed=3))
I'm feeding my cat with 3 cans of meat.
Please stick to the 'official' style guide for Python code:
https://www.python.org/dev/peps/pep-0008/
Useful software tools:
The Python language has some particular properties that might lead to some not-so obvious pitfalls.
a = 42
def f(b):
b = 11
f(a)
print(a)
42
The scalar variable is passed as copy.
a = [42]
def f(b):
c = b
c.append(11)
f(a)
print(a)
[42, 11]
The list a is passed as reference. Hence, b referes to the same object a in memory. The same applies for name changes of variables. c refers to the same object as b.
def f(a=[0]):
# Append a new element with the value as
# the maximum of all list values plus 1.
a.append(max(a) + 1)
return a
print(f())
[0, 1]
print(f())
[0, 1, 2]
The function changes the one-and-only list object that was instantiated as default keyword argument value.
From the NumPy webpage:
NumPy is the fundamental package for scientific computing with Python. It contains among other things:
- a powerful N-dimensional array object
- sophisticated (broadcasting) functions
- tools for integrating C/C++ and Fortran code
- useful linear algebra, Fourier transform, and random number capabilities
Besides its obvious scientific uses, NumPy can also be used as an efficient multi-dimensional container of generic data. Arbitrary data-types can be defined. This allows NumPy to seamlessly and speedily integrate with a wide variety of databases.
NumPy tutorial (pay special attention to the section about broadcasting rules):
https://docs.scipy.org/doc/numpy/user/quickstart.html
import numpy as np
Numpy arrays can be created from any iterable via the numpy.array() function:
a_list = [1, 2, 5, 3, 7, 10] # A list.
arr = np.array(a_list) # Instantiation of the numpy.ndarray class.
print(a_list*3) # Repeat list 3 times.
print(arr*3) # Multiply every array item with 3.
[1, 2, 5, 3, 7, 10, 1, 2, 5, 3, 7, 10, 1, 2, 5, 3, 7, 10] [ 3 6 15 9 21 30]
The array's data type can be specified:
arr = np.array(a_list, dtype=np.float64)
print(arr)
print(arr.dtype)
[ 1. 2. 5. 3. 7. 10.] float64
Numpy arrays have a dimensionality and a shape:
print(arr)
print(f'arr.ndim={arr.ndim}')
print(f'arr.shape={arr.shape}')
[ 1. 2. 5. 3. 7. 10.] arr.ndim=1 arr.shape=(6,)
Numpy arrays can be reshaped:
arr = arr.reshape((3,2))
print(arr)
print(f'arr.ndim={arr.ndim}')
print(f'arr.shape={arr.shape}')
[[ 1. 2.] [ 5. 3.] [ 7. 10.]] arr.ndim=2 arr.shape=(3, 2)
Numpy provides many math functions to be operated on an entire array:
arr = np.arange(3)
print(f'arr = {arr}')
print(f'np.sin(arr) = {np.sin(arr)}')
print(f'np.exp(arr) = {np.exp(arr)}')
print(f'np.power(10, arr) = {np.power(10, arr)}')
arr = [0 1 2] np.sin(arr) = [0. 0.84147098 0.90929743] np.exp(arr) = [1. 2.71828183 7.3890561 ] np.power(10, arr) = [ 1 10 100]
Numpy provides a few useful statistical functions:
arr = np. arange(10)
print(f'arr = {arr}')
print(f'np.mean(arr) = {np.mean(arr)}')
print(f'np.median(arr) = {np.median(arr)}')
print(f'np.std(arr) = {np.std(arr)}')
arr = [0 1 2 3 4 5 6 7 8 9] np.mean(arr) = 4.5 np.median(arr) = 4.5 np.std(arr) = 2.8722813232690143
Numpy provides several sorting algorithms via the numpy.sort function:
quicksort, heapsort, mergesort, timsort
unsorted_arr = np.array([3, 4, 2, 1, 6, 5])
sorted_arr = np.sort(unsorted_arr, kind='heapsort')
print(f'sorted_arr = {sorted_arr}')
sorted_arr = [1 2 3 4 5 6]
indices_ordering = np.argsort(unsorted_arr)
print(f'indices_ordering = {indices_ordering}')
indices_ordering = [3 2 0 1 5 4]
Two numpy arrays of different dimensions can be used in an expression by broadcasting the array elements of the smaller dimensional array by repeatedly prepending a dimension of length 1 to smaller array to allow element-wise operation:
import numpy as np
a = np.array([[1,2,3], [4,5,6]])
b = np.array([7,8,9])
c = a + b
print('shape of a:', a.shape)
print('shape of b:', b.shape)
print('shape of c:', c.shape)
print(c)
shape of a: (2, 3) shape of b: (3,) shape of c: (2, 3) [[ 8 10 12] [11 13 15]]
Elements of a numpy arrays can be accessed via indexing or slicing:
# Access by index (indices start with 0!):
print(a[0,2])
print(a[ [0,1,1], [0,1,2] ])
3 [1 5 6]
# Access by slice: start:end:step:
# Note: The start index is included, but the end index is excluded!
a = np.arange(0, 10)
print(a)
print(a[3:7])
print(a[3:7:2])
# Step can be negative to reverse the slice.
print(a[7:3:-2])
[0 1 2 3 4 5 6 7 8 9] [3 4 5 6] [3 5] [7 5]
A mask for an array is a boolean array selecting elements of an array.
a = np.arange(0, 5)
mask = np.array([True, False, False, True, True])
print(a)
print(a[mask])
[0 1 2 3 4] [0 3 4]
mask = (a == 0) | (a > 2)
print(mask)
print(a[mask])
[ True False False True True] [0 3 4]
Numpy supports arrays with field names, so called structured arrays.
dt = [('a', np.int8), ('b', bool), ('c', np.float32)]
arr = np.array([(3, True, 3.3), (2, False, 2.2), (4, False, 4.4)], dtype=dt)
print('arr.size =', arr.size)
print('arr.ndim =', arr.ndim)
# Retrieve a table row.
print('arr[1] =', arr[1])
arr.size = 3 arr.ndim = 1 arr[1] = (2, False, 2.2)
# Masking on columns.
mask = arr['a'] >= 3
print('mask =', mask)
print('arr[mask] =', arr[mask])
mask = [ True False True] arr[mask] = [(3, True, 3.3) (4, False, 4.4)]
# Setting column content.
arr['a'] = [42, 41, 40]
print(repr(arr['a']))
array([42, 41, 40], dtype=int8)
# Setting a table row. A tuple has to be used!
arr[1] = (7, True, 7.7)
print(arr)
[(42, True, 3.3) ( 7, True, 7.7) (40, False, 4.4)]
See numpy.org/doc/stable/reference/random/index.html for documentation.
# Create a random number generator with a seed.
rng = np.random.default_rng(42)
# Generate next random number(s) in range [0,1).
print(rng.random(size=5))
# Generate a numpy array with uniform random numbers in the range [1, 6].
a, b = 1, 6
rng.uniform(a, b, size=(5,2))
[0.77395605 0.43887844 0.85859792 0.69736803 0.09417735]
array([[5.87811176, 4.80569851],
[4.93032153, 1.64056816],
[3.25192969, 2.85399012],
[5.63382494, 4.2193256 ],
[5.11380807, 3.21707099]])
From the Matplotlib webpage:
Matplotlib is a Python 2D plotting library which produces publication quality figures in a variety of hardcopy formats and interactive environments across platforms.
# Call this magic function in the first line of your Jupyter notebook:
#%matplotlib notebook
# If that doesn't work, use
#%matplotlib widget
# Example: the final plot looks like this:
import matplotlib
fig = matplotlib.pyplot.figure(figsize=(6, 3))
# Split the figure into an array of fields: ncols x nrows.
grid = matplotlib.pyplot.GridSpec(ncols=1, nrows=1)
# Add a subplot by specifying column and row.
# You can also use slicing here for combining fields.
ax = fig.add_subplot(grid[0, 0])
# Input data
xval = np.linspace(0., 2.*np.pi, 101)
yval = np.sin(xval)
# Plot the input data:
# Combine the data points with a solid line.
ax.plot(xval, yval, "-", label="This is the sine function.")
[<matplotlib.lines.Line2D at 0x7f506aa4f7f0>]
# Axes formatting
ax.set_xlim(xval[0], xval[-1])
ax.set_ylim(-1., 1.)
(-1.0, 1.0)
# Set the major ticks location and format of the x-axis.
ax.xaxis.set_major_locator(matplotlib.ticker.FixedLocator([
0, np.pi/2, np.pi, 3/2*np.pi, 2*np.pi]))
ax.xaxis.set_major_formatter(matplotlib.ticker.FixedFormatter([
"$0$", r"$\frac{\pi}{2}$", r"$\pi$", r"$\frac{3\pi}{2}$",
r"$2\pi$"]))
# For linear ticks, Matplotlib is pretty good in figuring out the
# location of minor ticks.
ax.yaxis.set_major_locator(
matplotlib.ticker.LinearLocator(numticks=5))
ax.yaxis.set_minor_locator(
matplotlib.ticker.AutoMinorLocator())
# Labels
ax.set_xlabel(r"$\alpha$")
ax.set_ylabel(r"$\sin(\alpha)$")
Text(26.097222222222214, 0.5, '$\\sin(\\alpha)$')
# Legend
ax.legend(loc="upper right")
<matplotlib.legend.Legend at 0x7f506aa18640>
# Shrink the axes in order to fit in the labels.
grid.tight_layout(fig)
plt.show()
# Save the figure.
fig.savefig("example.pdf")
fig = plt.figure(figsize=(6, 3))
ax = fig.add_subplot(projection="mollweide")
plt.title("Mollweide Projection")
plt.grid(True)
plt.show()
Axes instance, like plot for plotting a point or line.# Draw a line.
fig = plt.figure(figsize=(6, 3))
ax = fig.add_subplot(projection="mollweide")
plt.title("Mollweide Projection")
plt.grid(True)
line = ax.plot(np.deg2rad([-90, 120]), np.deg2rad([-45, 60]))
plt.tight_layout()
plt.show()
pcolormesh
to plot it.# Create the data as a 2D numpy ndarray.
data = np.zeros((128, 64), dtype=np.float64)
data[54:74, 22:42] = np.reshape(np.random.normal(size=20*20), (20,20))
# Create mesh grid for the coordinates of the data bins.
lon = np.linspace(-np.pi, np.pi, data.shape[0])
lat = np.linspace(-np.pi/2, np.pi/2, data.shape[1])
(Lon, Lat) = np.meshgrid(lon, lat)
fig = plt.figure(figsize=(6,3))
ax = fig.add_subplot(projection="mollweide")
im = ax.pcolormesh(Lon, Lat, data.T, cmap=plt.cm.seismic,
vmin=-4, vmax=4)
plt.colorbar(im, label='random normal values')
plt.grid(True)
Commonly used (spherical) coordinate systems:
The astropy.coordinates module provides classes for coordinate transformations.
Documentation: docs.astropy.org/en/stable/coordinates
from astropy import units as u
from astropy.time import Time
from astropy.coordinates import SkyCoord, AltAz, EarthLocation
# Define location in sky.
TXS = SkyCoord(ra=77.358*u.degree, dec=5.693*u.degree, frame='icrs')
# Transformation into galactic coordinates.
print(TXS.galactic)
<SkyCoord (Galactic): (l, b) in deg
(195.40542828, -19.63626563)>
# Transform to local horizontal system at South Pole.
southpole = EarthLocation.from_geodetic(
lon=0*u.degree, lat=-90*u.degree)
TXS_local = TXS.transform_to(AltAz(
obstime=Time(56000, format='mjd'),
location=southpole))
print(TXS_local)
<SkyCoord (AltAz: obstime=56000.0, location=(3.91862092e-10, 0., -6356752.31424518) m, pressure=0.0 hPa, temperature=0.0 deg_C, relative_humidity=0.0, obswl=1.0 micron): (az, alt) in deg
(265.51023729, -5.70561117)>
# Transform to local horizontal system at Garching (CIP-Pool1) now.
import time
garching = EarthLocation.from_geodetic(
lat=48.26695746469105*u.degree, lon=11.67676875270207*u.degree)
TXS_local = TXS.transform_to(AltAz(
obstime=Time(time.time(), format='unix'),
location=garching))
print(TXS_local)
<SkyCoord (AltAz: obstime=1683273201.4854765, location=(4165583.55669134, 860889.61251078, 4736687.19293947) m, pressure=0.0 hPa, temperature=0.0 deg_C, relative_humidity=0.0, obswl=1.0 micron): (az, alt) in deg
(90.11878672, 7.7833946)>
From Wikipedia:
Object-oriented programming (OOP) is a programming paradigm based on the concept of "objects", which can
contain data and code: data in the form of fields (often known as attributes or properties), and code,
in the form of procedures (often known as methods).
In contrast to sequential global programming, object-oriented programming allows to encapsulate sets of functionalities into classes.
Classes can be instantiated as instances, i.e. objects of a particular class.
Example:
Given class Animal, the object cat and object dog could be instances of class Animal.
The class Animal can have data fields, i.e. properties, like name, and color.
_).__str__.# A not very useful class.
class MyClass(object):
pass
# MyClass is a class object!
# Let's look at its attributes:
dir(MyClass)
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']
# A simple class with a constructor, two attributes, and a
# special method.
class Direction(object):
r"""Directional vector in spherical coordinates.
Attributes
----------
azimuth : float
Azimuth angle in rad
zenith : float
Zenith angle in rad
"""
def __init__(self, azimuth, zenith):
"""This is the constructor method.
"""
self.azimuth = azimuth
self.zenith = zenith
def __str__(self):
"""This is a special class method to generate
output for `str(obj)`.
"""
return f"({self.azimuth}, {self.zenith})"
d = Direction(np.pi/3., np.pi/2.)
print("Direction:", d)
Direction: (1.0471975511965976, 1.5707963267948966)
class A(object):
r"""Class `A` with the property `a`,
which must be a positive number.
"""
def __init__(self):
self._a = 0
@property
def a(self):
r"""int: A positive number
"""
return self._a
@a.setter
def a(self, val):
if val >= 0:
self._a = val
else:
raise ValueError(
"Expect a positive number.")
a = A()
a.a = 42
print(a.a)
42
a.a = -1
--------------------------------------------------------------------------- ValueError Traceback (most recent call last) <ipython-input-78-9af0c80ce669> in <module> ----> 1 a.a = -1 <ipython-input-76-7eed1312c01a> in a(self, val) 17 self._a = val 18 else: ---> 19 raise ValueError( 20 "Expect a positive number.") ValueError: Expect a positive number.
Instance methods have the reference self to the class instance as first argument. By definition the constructor method is an instance method as well.
class A(object):
r"""A class with an instance method.
"""
def __init__(self, a):
self.a = a
a = A(42)
print(a.a)
42
The @classmethod decorator defines class methods. Class methods have a reference to a class object as first argument, i.e. a reference to the class object of the class instance itself. This allows for example to implement more than one constructor for a class.
class A(object):
r"""A class with class method.
"""
def __init__(self, a, b):
self.a = a
self.b = b
def __str__(self):
return "a = {o.a}, b = {o.b}".format(o=self)
@classmethod
def from_tuple(cls, t):
return cls(t[0], t[1])
a = A.from_tuple((1., 2.))
print(a)
a = 1.0, b = 2.0
The @staticmethod decorator defines static methods. Static methods can have no arguments at all. Static methods live inside the namespace of a class object. They can be called either from the class object or an instance of the class.
class Angle(object):
r"""A class with a static method
"""
def __init__(self, value=0.):
self.value = value
@staticmethod
def deg2rad(angle):
return angle * np.pi / 180.
print("30deg = {:.2f}rad".format(Angle.deg2rad(30.)))
30deg = 0.52rad
a = Angle()
print("30deg = {:.2f}rad".format(a.deg2rad(30.)))
30deg = 0.52rad
Python classes define magic methods for implementing operators, like +, -, *, etc.
See emulating numeric types for a full list of possible operators.
Binary arithmetic operators:
+: __add__(self, rhs)-: __sub__(self, rhs)*: __mul__(self, rhs)/: __div__(self, rhs)@: __matmul__(self, rhs)%: __mod__(self, rhs)&: __and__(self, rhs)|: __or__(self, rhs)Reflected (swapped) binary operators use __r<op>__(self, lhs), e.g. __radd__(self, lhs).
# Example for addition operator:
class Text(object):
def __init__(self, value):
self.value = value
def __add__(self, rhs):
return Text(self.value + str(rhs))
def __str__(self):
return self.value
t1 = Text('Hello')
t2 = Text(' ')
t3 = Text('World')
print(t1 + t2 + t3)
print(t1 + ' ' + t3)
print('Hello' + t2 + t3)
Hello World Hello World
--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-85-4a6eeadbd374> in <module> 14 print(t1 + t2 + t3) 15 print(t1 + ' ' + t3) ---> 16 print('Hello' + t2 + t3) TypeError: can only concatenate str (not "Text") to str
The last expression print('Hello' + t2 + t3) can be supported via a reflected add operator:
# Example for addition operator with reflected addition operator:
class Text(object):
def __init__(self, value):
self.value = value
def __add__(self, rhs):
return Text(self.value + str(rhs))
def __radd__(self, lhs):
return Text(str(lhs) + self.value)
def __str__(self):
return self.value
t2 = Text(' ')
t3 = Text('World')
print('Hello' + t2 + t3)
Hello World
>: __gt__(self, rhs)>=: __ge__(self, rhs)<: __lt__(self, rhs)<=: __le__(self, rhs)==: __eq__(self, rhs)!=: __ne__(self, rhs)True or False.__i<op>__(self, rhs), e.g. __iadd__(self, rhs).+=: __iadd__(self, rhs)-=: __isub__(self, rhs)*=: __imul__(self, rhs)The Python in-built function hasattr(obj, name) can be used to check if an object has an attribute of a given name.
angle = Angle(42)
print(hasattr(angle, 'deg2rad'))
True
The PEP8 Style Guide defines a propper naming scheme for classes, methods names and properties.
MyClass.print_a_number._), e.g. _a_private_number.In order to evolve classes in OOP, inheritance exists. Classes can be derived from base classes and inherit their properties and methods. This way code can be reused and development time can be saved.
super() function can be used to call methods of the parent class.class Animal(object):
"""This is the base class.
"""
def __init__(self, species, age):
self.species = species
self.age = age
# Instance method.
def description(self):
print(f'{self.species} is {self.age} years old')
@property
def age(self):
return self._age
@age.setter
def age(self, age):
if age >= 0:
self._age = age
else:
raise ValueError(
f'Provided {age} age has to be a positive number.')
class Dog(Animal):
"""The Dog class is derived from the base class `Animal`.
"""
def __init__(self, age):
# Call the constructor of the parent class.
# This first creates an animal object, which is then extended.
super().__init__('dog', age)
class Cat(Animal):
def __init__(self, age):
super().__init__('cat', age)
def description(self):
"""This methods overwrites the parent's description method.
"""
print(f'I am a {self.species}')
dog = Dog(5)
dog.description()
dog is 5 years old
cat = Cat(6)
cat.description()
I am a cat
print(type(cat))
print(type(cat.age))
print(type(cat.description))
<class '__main__.Cat'> <class 'int'> <class 'method'>
To check if an instance is of some class we can use the built-in function isinstance() which takes two arguments, an instance object and a class object and returns True if the given class is anywhere in the inheritance chain of the instance's class:
print(isinstance(cat, Cat))
print(isinstance(cat, Dog))
print(isinstance(cat, Animal))
True False True
We can also check if the given class is a subclass of another class with the built-in function issublass().
print(issubclass(Cat, Animal))
print(issubclass(Cat, Cat))
print(issubclass(Cat, Dog))
True True False
*args and keyword arguments **kwargs need to be passed on to the parent.class IsPuppy(object):
def __init__(self, *args, **kwargs):
# We need to call the parent's constructor
# to call the constructor of ALL base classes of a derived class.
super().__init__(*args, **kwargs)
def description(self):
print(f'I am a puppy.')
class Puppy(Dog, IsPuppy):
def __init__(self, age):
super().__init__(age)
puppy_dog = Puppy(1)
puppy_dog.description()
dog is 1 years old
Python determines which description to call using Method Resolution Order (MRO). We can check the order using the mro() method:
print(Puppy.mro())
[<class '__main__.Puppy'>, <class '__main__.Dog'>, <class '__main__.Animal'>, <class '__main__.IsPuppy'>, <class 'object'>]
Changing the MRO to call IsPuppy's description first.
class Puppy(IsPuppy, Dog):
def __init__(self, age):
super().__init__(age)
print(Puppy.mro())
puppy_dog = Puppy(1)
puppy_dog.description()
[<class '__main__.Puppy'>, <class '__main__.IsPuppy'>, <class '__main__.Dog'>, <class '__main__.Animal'>, <class 'object'>] I am a puppy.
abc package provides abstract base class functionality.
metaclass keyword with metaclass=abc.ABCMeta.@abstractmethod decorator.import abc
class AnimalBase(object, metaclass=abc.ABCMeta):
def __init__(self, species, age):
self.species = species
self.age = age
# Abstract instance method.
@abc.abstractmethod
def description(self):
# We use `pass` as a placeholder for the
# implementation by the derived class.
pass
# Define age property with abstract setter method.
@property
def age(self):
return self._age
@age.setter
@abc.abstractmethod
def age(self, age):
pass
Class derived from abc.ABCMeta cannot be instantiated unless all of its abstract methods and properties are implemented.
class Dog(AnimalBase):
def __init__(self, age):
super().__init__('dog', age)
dog = Dog(5)
dog.description()
--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-102-8c735f9acb69> in <module> ----> 1 dog = Dog(5) 2 dog.description() TypeError: Can't instantiate abstract class Dog with abstract methods age, description
Lets fix it by implementing the description and age methods.
class Dog(AnimalBase):
def __init__(self, age):
super().__init__('dog', age)
# Implement the description instance method.
def description(self):
print('I am the implementation of the description instance method.')
print(f'The {self.species} is of age {self.age}.')
# Implement the setter method of the age property.
@AnimalBase.age.setter
def age(self, age):
if age < 0:
print(f'Provided {age} age has to be positive number.')
self._age = None
else:
self._age = age
dog = Dog(-5)
dog.description()
Provided -5 age has to be positive number. I am the implementation of the description instance method. The dog is of age None.
dog = Dog(2)
dog.description()
I am the implementation of the description instance method. The dog is of age 2.
If a class implements a container, the iterator protocol can be used to iterate over the items of the container. See https://docs.python.org/3/library/stdtypes.html#iterator-types for details.
The interator protocol consists of two special instance methods: __iter__ and __next__.
The __iter__ instance method must return the iterator object, usually the class instance itself.
The iter() built-in function can be used to get an iterator of an object.
The __next__ instance method must return the next element of the container. If it raises the StopIteration exception, the iteration stops.
# Example of a container with the iterator protocol supported.
class MyBox(object):
def __init__(self, items):
self.items = items
def __iter__(self):
return MyBoxIterator(self)
# We define an iterator class that knows how to iterate through
# the items of the box.
class MyBoxIterator(object):
def __init__(self, box):
self.box = box
# The index attribute points to the next item in the box.
self.index = 0
def __iter__(self):
return self
def __next__(self):
# Check if we reached the end of the item sequence.
if self.index == len(self.box.items):
raise StopIteration()
self.index += 1
return self.box.items[self.index-1]
box = MyBox([8, 'a', 42, 'hello'])
items = [item for item in box]
print(items)
[8, 'a', 42, 'hello']
Our box class uses a sequence for storing the items. Sequences have iterators already implemented in Python, hence we can simplify the previous example by utilizing the iterator of the item sequence using the iter() built-in function.
class MySimpleBox(object):
def __init__(self, items):
self.items = items
def __iter__(self):
return iter(self.items)
box = MySimpleBox([8, 'a', 42, 'hello'])
for item in box:
print(item)
8 a 42 hello
Context managers in Python allow you to allocate and release resources precisely when you want to. The most widely used example of context managers is the with statement.
with open('myfile.txt', 'w') as f:
f.write('Hello World!')
This will open the file myfile.txt in write mode and enters a context. Once the block is finished, the context is exited. In this example the file will be closed automatically on exit.
Python implements context managers via two magic class methods:
__enter__(self)__exit__(self, type, value, traceback)class MyFile(object):
def __init__(self, filename, mode):
self.filename = filename
self.mode = mode
def __enter__(self):
print(f'Opening file {self.filename} in mode {self.mode}')
self.f = open(self.filename, self.mode)
return self.f
def __exit__(self, type, value, traceback):
print(f'Closing file {self.filename}')
self.f.close()
myfile = MyFile('myfile.txt', 'r')
with myfile as f:
print(f.readlines())
Opening file myfile.txt in mode r ['Hello World!'] Closing file myfile.txt
type, value, and traceback arguments of the __exit__ method allows the handling of exceptions.__exit__ method, the methods needs to return True.There are a couple of useful extra functions for loops (also called generators). The two most common generators are:
range(start (default=0), stop, stepsize (default=1)) -> creates a counting iterable, ie. 0, 1, 2, 3, ... stop. For a large number of iterations, range is much more memory-efficient than looping over the corresponding list of [0, 1, 2, ...], because the items are generated on the fly.enumerate(iterable) -> returns the index and the value of an iterableFor more info on where generators are useful and how to write your own generator functions, see e.g. https://realpython.com/introduction-to-python-generators/
A function which returns a generator iterator. It looks like a normal function except that it contains yield expressions for producing a series of values usable in a for-loop or that can be retrieved one at a time with the next() function.
def custom_range_generator(n):
i = 0
while i < n:
yield i
i += 1
range_generator = custom_range_generator(5)
print("next():", next(range_generator))
print("type:", type(range_generator))
print("list remaining numbers:", list(range_generator))
next(): 0 type: <class 'generator'> list remaining numbers: [1, 2, 3, 4]
# We have to create the generator again as we exhausted its items by calling
# `list` function earlier.
range_generator = custom_range_generator(5)
# Example in for loops
for i in range_generator:
print(i)
0 1 2 3 4
The pandas package provides userfriendly record arrays as DataFrame objects. 1D arrays are represented by Series objects.
import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3]})
print(df)
A 0 1 1 2 2 3
arr = pd.Series([1, 2, 3])
print(arr)
0 1 1 2 2 3 dtype: int64
# Show data types:
df.dtypes
A int64 dtype: object
# describe() shows a quick statistic summary of your data:
df.describe()
| A | |
|---|---|
| count | 3.0 |
| mean | 2.0 |
| std | 1.0 |
| min | 1.0 |
| 25% | 1.5 |
| 50% | 2.0 |
| 75% | 2.5 |
| max | 3.0 |
# Slicing rows.
df[0:2]
| A | |
|---|---|
| 0 | 1 |
| 1 | 2 |
Matplotlib has the module matplotlib.animation which provides functionality to create animated plots.
matplotlib.animation.FuncAnimation class can be used to animate a function of the form func(frame_number).Let's animate a point within a plot. The point will move forward on the x-axis.
# Import matplotlib and configure it so show animations in jupyter.
from matplotlib import pyplot as plt
from matplotlib import animation
# To visualize animations correctly, we use the jshtml backend.
from matplotlib import rc
rc('animation', html='jshtml')
# Define functions to plot a point and to prepare the animation.
def draw_point():
# Plot a blue point at (0,0). The plt.plot function returns
# a list of Line2D objects.
point = plt.plot(0, 0, marker='o', color='blue')[0]
return point
def prepare_animation():
point = draw_point()
def animate(frame_number):
# In each frame we update the x-coordinate of the
# Line2D object.
point.set_xdata([frame_number])
# The animate function needs to return a list of
# artists that will be redrawn.
return [point]
return animate
(fig, ax) = plt.subplots()
plt.title('Simple Animation')
plt.xlim(-1, 300)
# Define how long in seconds the animation should run.
length = 10
# Define how many frames per second we want to animate.
fps = 25
# Calculate the delay between frames in milliseconds.
# Keep in mind that we have a frame at the start AND
# at the end of the animation.
delay = int(1000 / (fps-1))
# Prepare the animation by plotting the point.
animation_func = prepare_animation()
# Animate the animation function.
ani = animation.FuncAnimation(
fig,
animation_func,
frames=int(fps * length), # Number of frames to animate.
interval=delay, # Delay between frames in milliseconds.
repeat=False,
blit=True # This will use less ressources.
)
ani
# To convert the animation into a HTML5 video one can do:
from IPython.display import HTML
HTML(ani.to_html5_video())

git?¶To clone a repository use the git clone <repo_url>.git <local_dir> command:
git clone https://github.com/icecube/skyllh.git skyllh
This copies (clones) the master branch of the repository from the remote location to the local disk.

To work on a new feature or bug fix, one should create a new branch from the master branch.
git checkout -b <branch> command.git clone https://github.com/icecube/skyllh.git skyllh.my_new_feature
cd skyllh.my_new_feature/
git checkout -b my_new_feature
After creating the new branch it lives in your local repository. In order to copy it to the remote repository one uses the git push --set-upstream origin <branch> command:
git push --set-upstream origin my_new_feature
To see on which branch you are currently working on use
git branch
to get the list of available branches.
After making code changes, these changes must be committed to the branch as a commit.
To see the changed files:
git status
To see the changes of a file:
git diff <file>
To stage a changed file for a commit, use the git add command:
git add <file>
To commit all staged files:
git commit -m "My commit description"
After a bug was fixed or a new feature was implemented the development branch needs to be merged with the master branch.

git checkout master
git fetch
git pull
git merge my_new_feature
git branch -d my_new_feature
git merge will fail before creating the merge commit.git status will tell where merge conflics are.here is some content not affected by the conflict
<<<<<<< master
this is conflicted text from master
=======
this is conflicted text from my_new_feature branch
>>>>>>> my_new_feature;
git add.git commit.
pip install scalene%load_ext scalene
Scalene extension successfully loaded. Note: Scalene currently only supports CPU+GPU profiling inside Jupyter notebooks. For full Scalene profiling, use the command line version.
%%scalene
# Profile more than one line of code in a cell
x = 0
for i in range(1000):
for j in range(1000):
x += 1
Works on the command line:
scalene --reduced-profile --- my_program.py --my_option value
Your program's command line comes after the three dashes (---)
Produces and displays a webpage with the profiling information
unittest package implementing a unit test framework.import unittest
class TestStringMethods(unittest.TestCase):
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello', 'world'])
# check that s.split fails when the separator is not a string
with self.assertRaises(TypeError):
s.split(2)
#if __name__ == '__main__':
# unittest.main()
unittest.TestCase base class.test.assert methods are used to ensure correct variable values.This notebook can be found on github.com:
https://github.com/martwo/teaching/tree/master/WS_2022_23/advanced_python
and can be cloned via:
git clone https://github.com/martwo/teaching.git
Converting this notebook into slides and launching a browser window opening them:
jupyter nbconvert advanced_python.ipynb --to slides --post serve